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DETAILED ACTION 

1 . This action is responsive to communications: amendment filed 10/13/2005, to the original application 
filed 2/6/2002, drawings filed 3/25/2002. The present application is a continuation (filed under 37 CFR 1.53(b)) 
of parent case 08/900,421 (now abandoned). Said parent case filed 7/25/1997. 

2. Claims 1, 3-30 pending. Claims 26-30 have been added. Claims 1, 14, 16, 19, 20, 26-30 are independent 
claims. 

Claim Rejections - 35 USC§ 112 

3. The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making and using 
it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with 
which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by the 
inventor of carrying out his invention. 

4. Claims 1, 3-25 are rejected under 35 U.S.C. 112, first paragraph, as failing to comply with 
the enablement requirement. The claim(s) contains subject matter which was not described in the 
specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most 
nearly connected, to make and/or use the invention. 

Regarding claims 1, 3-25, independent claims 1, 14, 16, 19, 20 are newly amended to recite 
"multicasting separately the two streams", "transmitting through separate streams", or "received as separate 
streams of data", etc. (each said independent claim is similarly amended). The examiner cannot find specific 
support for these amendments in Applicant's specification. 
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5. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject 
matter which the applicant regards as his invention. 

6. Claims 1, 3-25 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

In regard to claims 1, 3-25, independent claims 1, 14, 16, 19, 20 are newly amended to recite 
''multicasting separately the two streams", "transmitting through separate streams", or "received as separate 
streams of data", etc. (each said independent claim is similarly amended). Since there are many ways to 
send/receive separate streams of data, and since Applicant's specification sheds no descriptive light on this 
limitation (see above), Applicant's newly amended subject matter is vague and indefinite. 

For the purposes of art rejection, the examiner will not address said subject matter until the above issues 
are resolved. 

Claim Rejections - 35 USC § 103 
7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the 
differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall 
not be negatived by the manner in which the invention was made. 
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Art Unit: 2176 

8. Claims 1, 6-13, 16-17, 19, 21, 23-24, 26, 28, 29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over M endelson et al. (hereinafter Mendelson), U.S. Patent No. 5,754,783 issued May 1998, 
in view of Dunvard et al. (hereinafter Durward), U.S. Patent No. 5,950,202 issued September 1999. 



In regard to independent claims 1, and 26, Mendelson teaches interleaving (i.e. synchronizing) timed 
primary and untimed secondary data streams at a server, said resulting stream(s) eventually transported to clients 
for display (Mendelson Abstract, column 4 lines 18-22, column 8 lines 15-20). Mendelson defines timed 
primary content data as games, movies, audio music, etc., acquired and stored in a storage device (hard drive) on 
said server (Mendelson column 1 lines 13-22, column 6 lines 8-11, Figure 4 item 421). Mendelson defines 
untimed secondary content as video text, graphic images, private data such as files, databases, or stuffing bits, 
etc. (Mendelson column 1 lines 24-28, column 3 lines 50-54, column 6 lines 23-26, Figure 4 item 414). It is 
noted that any number of data streams can be utilized (Mendelson column 4 lines 54-58). 

Mendelson does not specifically teach motion ''capture" data. However, Durward teaches a virtual 
reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since communication of 
changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion data stream is 
captured accordingly. It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply Durward to Mendelson, providing the benefit of motion data capture for a realistic virtual reality 
experience. 

Mendelson does not specifically disclose "motion" type data. However, Mendelson' s secondary content 
(video text/images etc.) can at least be interpreted as suggestive of motion data since it is displayed as a 
video/movie type stream. In addition, Mendelson's secondary content can also comprise "padding", null/stuffing 
data, interleaved via a stuffing buffer, helping to preserve the quality (i.e. motion - wander, jitter, etc.) of the 
timed primary data (Mendelson column 2 lines 50-67, column 3 lines 9-20, 50-54, column 8 lines 14-17, Figure 
4 item 440). It would have been obvious to one of ordinary skill in the art at the time of the invention to interpret 
Mendelson's secondary data as a type of "motion" data, providing the benefit of controlling/minimizing possible 
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timed motion related problems (i.e. wandering, jittering, etc.) of the primary content (compare the above two 
paragraphs with claim 1 "A method of synchronizing asynchronous time-based and motion data in a system in 
which the time-based data and the motion data are transmitted by a server over a network to a client, the 
method comprising", and "retrieving a time-based data stream and a motion data stream at the server..") 

Mendelson teaches receiving various time-based and motion data at a server, as explained above. In 
addition, since Mendelson discloses typical timed and untimed content within a frames environment (Mendelson 
column 1 lines 17-18, 24-25), combined with Mendelson's teaching of using the MPEG standard (typically 
applied to MPEG files, which are frame-based) (Mendelson column 4 lines 48-52), Mendelson's data streams 
can be interpreted as frame based (compare with claim 1 "... each stream comprising frames of data"). 

Mendelson teaches variable buffering (at a server) by measuring the bit rate stream, and buffers via the 
use of variable usage (interleaving) of primary, secondary and stuffing content buffers (Mendelson column 6 
lines 62-67, column 8 lines 6-20, Figure 4 items 440, 450, 460). It is noted that each type of content has its own 
buffer, and are regulated accordingly. It is noted that Mendelson's resulting data stream can be interpreted as 
comprising at least two interleaved streams (now custom synchronized) of various data types. In addition, 
multiple elementary streams within a transport stream can be utilized (Mendelson column 4 lines 54-63, column 
6 lines 4-8) (compare with claim 1 "variable buffering one of the time-based data stream and the motion data 
stream at the server to produce two streams having synchronized frames"), 

Mendelson teaches transport of a final (custom synchronized) stream(s) to customers (clients) for 
eventual playback (Mendelson column 6 lines 4-8, column 8 lines 10-12; compare with claim 1 "using the 
synchronized frames at the client for playback of synchronized motion and time-based data to a user."). 

In regard to dependent claim 6, Mendelson teaches secondary data as stuffing data, or as private data 
such as files or databases (see rejection of claim 1, also Mendelson column 6 lines 24-26). Mendelson does not 
specifically teach said motion data mapped to control movement of a virtual figure displayed in a scene at the 
client. However, Durward teaches a method whereby updated positional data from a person's head position 
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sensor is mapped and used to determine the position of a virtual being defined for that user, communicating 
graphical data to the user via database data (Durward, column 6 lines 29-32, 46-52, column 7 lines 12-20). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply the virtual 
position database data of Durward to Mendelson's secondary database data, providing Mendelson increased 
spatial imagery to the multimedia stream. 

In regard to dependent claim 7, Mendelson does not specifically teach using a body suit for 
generating motion data. However, Durward teaches a method of sensing position data with the use of a body suit 
(Durward, column 3 line 25-33). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to apply Durward to Mendelson, providing Mendelson the benefit of three dimensional physical 
contour mapping, providing increased spatial image accuracy to Mendelson's video stream method. 

In regard to dependent claim 8, Mendelson teaches primary content (video, audio data, etc.) as well as 
secondary content (video text, images, files, database data, along with "stuffing" data, etc.) as explained in the 
rejection of claim 1. As the resulting interleaved stream(s) is played back on a client, any combination of 
foreground/background combination can be typically specified, (i.e. the primary data (i.e. a movie scene) can be 
run as background to the secondary/motion data controlling it, or run as background to any related database files 
in the foreground, etc). 

In regard to dependent claims 9, 10, Mendelson teaches buffering/processing occurring at the same 
time the program is being transported to customer equipment (Mendelson column 8 lines 9-12; compare with 
claim 9). 

Mendelson teaches audio (i.e. voice) data (Mendelson column 4 lines 57-59; compare with claim 10). 

In regard to dependent claim 11, Mendelson teaches an embodiment using streams associated with 
data channels (Mendelson column 6 lines 1-8). Mendelson also teaches a header packet (a descriptor packet), 
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said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach that a client may join in progress a multicasting of a stream. 
However, Durward teaches virtual reality presentations where a number of contributing user's data is collected 
and analyzed accordingly, so a to facilitate multicasting (and interaction between figures) of the virtual world to 
various users (Durward Figures 4-6, 7 column 8 lines 5-15). Durward also teaches users logging on to a system 
via modems, said users data is shared with other users and continuously updated so as to create a virtual reality 
world (Durwald, column 8 lines 22-24, 41-64, Figure 1). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to apply Durward to Mendelson, providing Mendelson the benefit of 
multicasting associated with synchronization of streams. 

In regard to dependent claims 12, 13, Mendelson teaches buffering of streams comprising buffering of 
primary and secondary content (said primary content can be audio data, Mendelson column 4 lines 57-59), along 
with necessary interleaved motion type data (secondary/stuffing data), producing a sequenced synchronized 
stream regarding constant verses variable bit stream rate (see also rejection of claim 1, also Mendelson column 6 
lines 62-67). 

In regard to independent claims 16 and 28, claims 16 and 28 reflect the apparatus used to perform the 
methods comprising computer readable instructions as claimed in claim 1, and in further view of the following, 
is rejected along the same rationale. 

Mendelson does not specifically teach motion "three dimensional" data stream. However, Durward 
teaches a virtual reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since 
communication of changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion 
data stream is captured accordingly and is associated with 3D data. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing the benefit of 
motion data 3D capture for a realistic virtual reality experience. 
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In regard to dependent claim 17, Mendelson does not specifically teach an apparatus whereby 
synchronized motion/time-based data is milticasted to clients in a network. However, Durward teaches an 
apparatus whereby users log on to a system via modems, said users data is shared with other users and 
continuously updated so as to create a virtual reality world (Durward, column 8 lines 22-24, 41-64, Figure 1). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply the data sharing 
method of Durward to Mendelson, providing Mendelson the benefit of continuous updating and increased 
uniform user awareness in a virtual environment. 

In regard to independent claims 19 and 29, claims 19 and 29 incorporate substantially similar subject 
matter as claimed in claim 1, and in further view of the following, is rejected along the same rationale. 

Mendelson does not specifically teach motion "capture" data. However, Durward teaches a virtual 
reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since communication of 
changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion data stream is 
captured accordingly. It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply Durward to Mendelson, providing the benefit of motion data capture for a realistic virtual reality 
experience. 

Mendelson teaches secondary data as stuffing data, or as private data such as files or databases (see 
rejection of claim 1, also Mendelson column 6 lines 24-26). Mendelson does not specifically teach said motion 
data mapped to control movement of a virtual figure displayed in a scene at the client, along with 
synchronization. However, Durward teaches a method whereby updated positional data from a person's head 
position sensor is mapped and used to determine the position of a virtual being defined for that user, 
communicating graphical data to the user via database data (Durward, column 6 lines 29-32, 46-52, column 7 
lines 12-20). Since the playback of Mendelson's resulting stream synchronizes the secondary data with the 
primary content, the primary data (i.e. video) can be synchronized with added secondary database data (related 
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to a virtual figure). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
apply the virtual position database data of Durward to Mendelson' s secondary database data, providing 
Mendelson increased spatial imagery and desired synchronization to the resulting multimedia stream. 

In regard to dependent claim 21, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 

In regard to dependent claim 23, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 

In regard to dependent claim 24, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 
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9. Claim 3 is rejected under 35 ILS.C. 103(a) as being unpatentable Mendelson and Durward as 
applied to claim 1 above, and further in view of Baumgartner et al. (hereinafter Baumgartner), U.S. 
Patent No. 5,642,171 issued June 1997. 

In regard to dependent claim 3, Mendelson teaches buffering based on a calculated difference (delay) 
between a variable bit stream, and a constant bit stream (Mendelson column 6 lines 62-67). Mendelson does not 
specifically teach a difference between the delay for motion and time-based data stream to determine buffering 
for a faster stream. However, Baumgartner teaches a method whereby a current video frame number is 
subtracted from a current audio frame number to determine if the audio is too far ahead of the video 
(Baumgartner column 13 lines 60-67, column 14 lines 1-2). It would have been obvious to one of ordinary skill 
in the art at the time of the invention to apply the difference calculation method of Baumgartner to Mendelson' s 
stream analysis, providing Mendelson the benefit of more accurate time synchronization to the server buffer 
method disclosed by Mendelson. 

10. Claims 4, 14-15, 18, 22, 27 are rejected under 35 U.S.C 103(a) as being unpatentable over 
Mendelson and Durward, and further in view of Shibata, Y., Media synchronization protocols for packet 
audio-video system on multimedia information networks (hereinafter Shibata), IEEE; January 3-6, 1995 
pp.594-601. 

In regard to dependent claim 4, Mendelson does not specifically teach transfer of data values for a 
frame occuring only for a frame that has changed since a last frame was transmitted. However, Shibata discloses 
a method whereby audio data is sent from the video server to the client station only during a "talk spurt", with 
constant frame rates occurring during periods of audio silence (Shibata p.597, section 4.3 "Silence detected 
synchronization"). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
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apply Shibata to Mendelson's stream interleaving, providing Mendelson the benefit of file size and transmission 
reduction (streamlining) of the audio/video data streams. 

In regard to independent claims 14 and 27, claims 14 and 27 incorporate substantially similar subject 
matter as claimed in claim 1, and in further view of the following, is rejected along the same rationale. 

Mendelson does not specifically teach motion "three dimensional" data stream. However, Durward 
teaches a virtual reality database for coordinating a three-dimensional virtual space (Durward Abstract). Since 
communication of changed reference motion data is essentially real-time (Durward Figures 1, 4), said motion 
data stream is captured accordingly and is associated with 3D data. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing the benefit of 
motion data 3D capture for a realistic virtual reality experience. 

Mendelson teaches an embodiment using streams associated with "data channels" (Mendelson column 
6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), said header typically used to describe 
data within transport stream data (i.e. each 188 byte packet) (Mendelson column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach transfer of data values (channels) for a occuring only for changed 
data values. However, Shibata discloses a method whereby audio data is sent from the video server to the client 
station only during a "talk spurt", with constant frame rates occurring during periods of audio silence (i.e. a talk 
spurt changes the data values) (Shibata p.597, section 4.3 "Silence detected synchronization"). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to apply Shibata to Mendelson's 
stream interleaving, providing Mendelson the benefit of file size and transmission reduction (streamlining) of 
the audio/video data streams. It is to be noted that in order to adjust frame rates as above, the server must know 
that a change in the data has occurred, which is typically achieved by comparing new data (the talk spurt) with 
the last recognized data value (previously recognized as empty - reflecting a period of silence). 



Application/Control Number: 10/068,829 
Art Unit: 2176 



Page 12 



In regard to dependent claim 15, Mendelson teaches an embodiment using streams associated with 
data channels (Mendelson column 6 lines 1-8). Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

In regard to dependent claim 18, Mendelson teaches an embodiment using streams associated with 
"data channels" (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach transfer of data values (channels) for a occuring only for changed 
data values. However, Shibata discloses a method whereby audio data is sent from the video server to the client 
station only during a "talk spurt", with constant frame rates occurring during periods of audio silence (i.e. a talk 
spurt changes the data values) (Shibata p.597, section 4.3 "Silence detected synchronization"). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to apply Shibata to Mendelson' s 
stream interleaving, providing Mendelson the benefit of file size and transmission reduction (streamlining) of 
the audio/video data streams. It is to be noted that in order to adjust frame rates as above, the server must know 
that a change in the data has occurred, which is typically achieved by comparing new data (the talk spurt) with 
the last recognized data value (previously recognized as empty - reflecting a period of silence). 

In regard to dependent claim 22, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 
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1 1 . Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson and Durward, 
as applied to claims 1, above, and further in view of Katseff et al. (hereinafter Katseff), U.S. Patent No. 
5,822,537 issued October 1998. 

In regard to dependent claim 5, Mendelson teaches a networking environment (Mendelson column 4 
lines 28-33). Mendelson does not specifically teach using the Internet for distribution. However, Katseff teaches 
an internet as an example of suggested types of networks that can be used (Katseff, column 3 lines 58-63). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply Katseff to 
Mendelson, providing Mendelson the benefit of a widespread and familiar network. 

12. Claims 20, 25, 30 are rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson, 
Durward, and Baumgartner. 

In regard to independent claims 20 and 30, claims 20 and 30 incorporate substantially similar subject 
matter as claimed in claims 1, 3, 10, 1 1, and are rejected along the same rationale. 

In regard to dependent claim 25, Mendelson does not specifically teach motion "sensor" data. 
However, Durward teaches a virtual reality database for coordinating a three-dimensional virtual space 
(Durward Abstract). Since communication of changed reference motion data is essentially real-time via sensor 
helmets (Durward Figures 1, 4), said motion data stream is captured via sensors accordingly. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply Durward to Mendelson, providing 
the benefit of motion data capture for a realistic virtual reality experience. 



Application/Control Number: 10/068,829 Page 14 

Art Unit: 2176 

Response to Arguments 

13. Applicants arguments filed 10/13/2005 have been fully and carefully considered but they are not 
persuasive. 

During the interview conducted 9/28/2005 (Interview summary included with this action), the examiner 
indicated that (upon cursory perusal) Applicant's proposed amendments appear to overcome the cited art of 
record. However, the examiner cannot find specific support for this amendment in the specification at this time. 
Although the examiner cannot recall if any enablement issues were discussed during said interview, Applicant 
was respectfully reminded of this during the previous interview conducted 5/5/2004 (mailed 5/1 1/2004). 



Conclusion 

14. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth 

in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the 
mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this 
final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no 
event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this 
final action. 

15. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to William L. Bashore whose telephone number is (571) 272-4088. The examiner can normally be 
reached on 1 1 :30am - 8:00pm EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Heather 
Herndon can be reached on (571) 272-4136. The fax phone number for the organization where this application 
or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free). 
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WILLIAM BASHORE 
PRIMARY EXAMINER 

January 6, 2006 



